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Process Product Integrity Audits / / ^ - 

A Hardware Auditing Technique For the "90s" 

by Mike Taylor 

Early in 1993, the Space Shuttle program experienced hardware problems that delayed several shuttle launches 
and missions. NASA review determined that these problems could have been prevented. NASA further 
concluded that a new kind of Quality emphasis at all Space Shuttle prime contractors and subcontractors was 
necessary to ensure mission success. To meet this challenge, NASA initiated an innovative review process 
called Process/Product Integrity Audits (PPIA). NASA's initial application of a PPIA at a prime contractor 
revealed some unexpected problems in the manufacture of Shuttle hardware such as: 
o Floor practitioners* did not in all cases follow the planning documents 

("Practitioner" is a term coined for PPIA's meaning any employee who actually touches or builds hardware.) 
o Planning documents were frequently inadequate 
o Barriers existed to make planning changes 

o Hardware inspections were at times inadequate because of the ill advised belief that someone else in the 
system would detect errors. 

o Lack of understanding of the practitioner or the inspectors' stamps as product warranty, 
o Adequate skill training was not necessarily available across a product line 

Martin Marietta Manned Space Systems in New Orleans, La. evaluated NASA’s initial PPIA technique and 
results. Manned Space Systems decided it would take the initiative and voluntarily make use of the PPIA's to 
validate its product, the Space Shuttle External Tank. 

Manned Space Systems recognized immediately that the PPIA was a new breed of audit. It was an audit that 
focused upon the ability of each practitioner to perform and then to warrant his or her work. Standard audits only 
verify system compliance to a set of requirements. The "up close and personal" emphasis upon an employee by 
employee warranty was a new concept, and putting it into practice produced some surprises. 



When we completed our PPIA, we knew that we had several of the problems identified during the initial NASA 
review described above. None of the audit problems identified would have affected the function and integrity of 
our hardware or compromised Mission Success. However, some of the problems, if left undiscovered and 
unchecked could have caused unnecessary delays and higher costs of production, as well as undesirable results 
during testing. We reasoned that if this was true of our company, it was very likely true at our suppliers. 

We described our experience with the PPIA to our suppliers, anticipating that once exposed to the results of the 
new audit technique, they would volunteer, as we had, to validate their own systems and hardware. Our plan was 
then to implement the same level and degree of PPIA on our suppliers as we had experienced. We informed our 
suppliers that to perform a PPIA correctly, three distinct phases of audit were required: 

o Phaser Fact Finding . This is performed by a team or series of teams of practitioners. 

0 Phase II Findings Review . This phase is performed by an objective group of reviewers other than the 

Phase I practitioners. 

o Phase III Corrective Action Implementation . This phase is accomplished by the audited company and 

monitored by customer members of the audit teams. 

•Audit phases are described in panels A, B and C 

Several of our suppliers did. in fact, step up to Ihe PPIA challenge. Kaman Aerospace, Moosup, CT and 
Ducommun Corporation (Aerochem) Orange, CA, were the first of our 33 major suppliers of Space Shuffle 

External Tank hardware to volunteer. 

Each is a medium size company with diverse product lines. The experience of each company was amazingly 


similar: 



o Systems producing hardware were validated at each of the companies. Although a moderate number of 
findings were discovered, as in the Manned Space Systems experience, none was serious enough to 
compromise the hardware or Mission Success 

o Practitioners praised the PPIA and commended each company's management for their willingness to, 
"listen to what the floor says", and make suggested changes to the system, 
o Each company was intrigued by the new audit concepts, particularly: 

- Stamp Warranty - the professional guarantee that the operation signed off was performed as exactly 
stated in the planning and the location of the operation in the process flow was correct. 

- Potential for easily adapting the PPIA techniques to other product lines. 

- Depth of the audit - looks at the system from the Procurement paperwork through shipment of 
hardware from the supplier's dock. 

- Teamwork between all interna l/extemal customers necessary to perform a PPIA. 

Potential to augment the bottom line through productivity and efficiency improvements. 

- Power to creatively harness and utilize the specific knowledge of the people who actually build and 
inspect the hardware. 

In conclusion, the initial industry impression of the PPIA was that it could be burdensome. To the contrary, all 
companies that accepted the challenge of the PPIA have stated that they experienced immediate benefits from 
the audit and anticipate that the benefits will compound in the years ahead. 



Panel A 


Job instructions are reviewed by practitioners who evaluate if operations described can be warranted as 
meeting design requirements. 

Reviews are very candid, each practitioner encouraged to highlight concerns without fear of retribution. 
Suppliers will follow the flow of several part numbers from Purchase Agreement receipt to shipment. 
Detailed checklists are developed to test each aspect of the system. 

All issues raised are documented for root cause analysis and corrective actions. 

Findings are then categorized from Category 1 to Category 3, with Category 1 finding the most severe 
(usually detrimental to form, fit, function of the hardware) and Category 3 being the least severe. 



Panel B 


Phase II - Management Review 

o Determine if Phase I results covered agreed upon areas. 

o This phase is dynamic and requires a plan that identifies individuals and methods. 

o Phase I team members are not permitted on the Phase II analysis team. Phase II team should include 
systems experts and external customers. 

o Phase II team analyzes all Phase I findings, reviews the categorization of the findings by the Phase I team 
and approves corrective actions. 

o Phase II team may be chartered to review a specific component or an entire process. If the Phase II team 
actually does perform audits, the auditing methodology from Phase I is applied. 

o Phase II concludes with a report and generally a presentation by each of the auditing teams to company 
management and all external customers. The minimum subjects in the report/briefing are: 

Phase II team activities and charters 

Findings agreed upon by the teams 

Master Schedule for corrective actions/implementation. 



Panel C 


phase Hi - C orrective Action 

o All corrective actions for findings identified in Phases I & II are implemented during Phase III. 

o Phase III is generally much longer in duration than the earlier phases due to the nature of implementing the 

necessary detailed changes. 

o Corrective actions are universally prioritized, with Category I items highest on the list, 

o Companies are usually asked to commit to a period ol time (six months to a year) to implement corrective 

action. 

o Findings can and should be worked in unison or combination wherever possible or reasonable, 

o Documentation systems or metrics should be applied to track corrective actions, especially those affecting 

productivity or efficiency. 

o Quarterly reports regarding findings close-out activity are generated with a final close-out report to complete 
the audit. 




